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Abstract 

Objectives To develop prediction models that better estimate the pretest 
probability of coronary artery disease in low prevalence populations. 

Design Retrospective pooled analysis of individual patient data. 

Setting 18 hospitals in Europe and the United States. 

Participants Patients with stable chest pain without evidence for previous 
coronary artery disease, if they were referred for computed tomography 
(CT) based coronary angiography or catheter based coronary 
angiography (indicated as low and high prevalence settings, respectively). 

Main outcome measures Obstructive coronary artery disease (>50% 
diameter stenosis in at least one vessel found on catheter based coronary 
angiography). Multiple imputation accounted for missing predictors and 
outcomes, exploiting strong correlation between the two angiography 
procedures. Predictive models included a basic model (age, sex, 
symptoms, and setting), clinical model (basic model factors and diabetes, 
hypertension, dyslipidaemia, and smoking), and extended model (clinical 
model factors and use of the CT based coronary calcium score). We 
assessed discrimination (c statistic), calibration, and continuous net 
reclassification improvement by cross validation for the four largest low 
prevalence datasets separately and the smaller remaining low prevalence 
datasets combined. 

Results We included 5677 patients (3283 men, 2394 women), of whom 
1634 had obstructive coronary artery disease found on catheter based 
coronary angiography. All potential predictors were significantly 
associated with the presence of disease in univariable and multivariable 
analyses. The clinical model improved the prediction, compared with 
the basic model (cross validated c statistic improvement from 0.77 to 
0.79, net reclassification improvement 35%); the coronary calcium score 
in the extended model was a major predictor (0.79 to 0.88, 102%). 
Calibration for low prevalence datasets was satisfactory. 

Conclusions Updated prediction models including age, sex, symptoms, 
and cardiovascular risk factors allow for accurate estimation of the pretest 
probability of coronary artery disease in low prevalence populations. 
Addition of coronary calcium scores to the prediction models improves 
the estimates. 

Introduction 

In the United States, about 10.2 million people have chest pain 
complaints each year, 1 and more than 1.1 million diagnostic 



procedures of catheter based coronary angiography are 
performed on inpatients each year. 2 In a recent report based on 
the national cardiovascular data registry of the American College 
of Cardiology, 3 only 41% of patients undergoing elective 
procedures of catheter based coronary angiographies are 
diagnosed with obstructive coronary artery disease. The report' s 
authors concluded that better risk stratification was needed, 
underlined by decision analyses showing that the choice of 
further diagnostic investigation in patients with chest pain 
depends primarily on the pretest probability of coronary artery 
disease. 4 " 6 

The American College of Cardiology/ American Heart 
Association, 7 8 European Society of Cardiology, 9 and United 
Kingdom 1 " currently recommend using the Diamond and 
Forrester model" or the Duke clinical score 12 13 to estimate the 
pretest probability of coronary artery disease in patients with 
chest pain. The Diamond and Forrester model tends to 
overestimate the probability of coronary artery disease (defined 
as >50% stenosis), and a revised version has recently been 
published. 14 The Duke clinical score 12 13 estimates the probability 
of coronary artery disease (>75% stenosis) which, to our 
knowledge, has not been validated in populations outside the 
US. Although the American College of Cardiology/ American 
Heart Association and European Society of Cardiology 
recommend exercise electrocardiography to select patients for 
further diagnostic investigation, UK guidelines recommend 
using the computed tomography (CT) based coronary calcium 
score in patients with a low to intermediate pretest probability 
(10-29%). 

We perceived a need for an updated and stepwise approach to 
estimate the probability of coronary artery disease in patients 
with new onset of chest pain in a low prevalence population as 
clinical information and test results become available, in 
particular because implementation of the guidelines needs 
calculation of the pretest probability. Therefore, we aimed to 
estimate the probability of obstructive coronary artery disease 
on the basis of clinical presentation and cardiovascular risk 
factors, and to determine the incremental diagnostic value of 
exercise electrocardiography and the coronary calcium score. 
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Methods 
Design overview 

Researchers from Europe and the US formed a consortium. An 
existing database of at least 80 eligible patients was required 
for participation. Participation did not involve any financial 
incentives. All patients had to be enrolled in single centre 
studies, and local approval from the institutional review board 
for the original research objectives was required. The consortium 
is part of the European network for the assessment of imaging 
in medicine, which is an initiative of the European Institute of 
Biomedical Imaging Research. 15 One of the network's goals 
was to perform pooled analyses of existing prospectively 
collected data, improving power and increasing generalisability 
of results. 

Participants 

Patients were eligible for the analysis if they presented with 
stable chest pain and were referred for catheter based or CT 
based coronary angiography (>64 slice). Patients were not 
eligible if they had acute coronary syndrome or unstable chest 
pain, had a history of myocardial infarction or previous 
revascularisation (percutaneous coronary intervention or 
coronary artery bypass graft surgery), or provided no informed 
consent. For the diagnosis of coronary artery disease, catheter 
based coronary angiography is regarded as the reference 
standard, which is an expensive and invasive procedure with a 
risk of complications. Non-invasive testing is generally 
recommended to select patients who might benefit from catheter 
based coronary angiography. CT based coronary angiography 
is a less invasive and less expensive test, with a high sensitivity 
and specificity for the detection of coronary artery disease on 
catheter based coronary angiography. 16 " 18 A negative result from 
CT based coronary angiography virtually excludes the presence 
of obstructive coronary artery disease, whereas a positive result 
might need confirmation by the catheter based test. 

Fourteen datasets consisted of consecutive patients enrolled in 
a prospective study for other research objectives. Four datasets 
consisted of patients retrospectively identified as eligible via 
electronic radiology reporting systems. Inclusion and exclusion 
criteria were evaluated by experienced doctors and missing 
information was obtained from patient records (web appendix 
table l)."- 21 

Definitions 

We collected data for age, sex, symptoms, cardiovascular risk 
factors, test results, and presence of coronary artery disease. 
Chest pain symptoms were classified as typical, atypical, or 
non-specific. Typical chest pain was defined as all of the 
following criteria: (1) substernal chest pain or discomfort that 
is (2) provoked by exertion or emotional stress and (3) relieved 
by rest or nitroglycerine (or both). We defined atypical chest 
pain as two of these criteria. If one or none of the criteria was 
present, symptoms were classified as non-specific. 22 

Definitions for hypertension, diabetes, dyslipidaemia, and 
smoking differed slightly across hospitals (web appendix table 
1). Table 1JJ lists the most common definitions. We determined 
coronary calcium scores by the Agatston method 23 and used log 
transformation to account for its skewed distribution. 

Outcomes 

Primary outcome was obstructive coronary artery disease, 
defined as at least one vessel with at least 50% diameter stenosis 
found on catheter based coronary angiography. Since we 



combined existing databases from different hospitals, CT based 
and catheter based coronary angiographies were performed at 
each institution according to local protocols; we allowed both 
visual assessment and quantitative assessment for the 
interpretation of results for these procedures. 

Statistical analysis 

We assumed missing data occurred at random, depending on 
the clinical variables and the results of CT based coronary 
angiography, and performed multiple imputations using chained 
equations. 24 Missing values were predicted on the basis of all 
other predictors considered, the results of CT based coronary 
angiography, as well as the outcome. 25 26 We created 20 datasets 
with identical known information, but with differences in 
imputed values reflecting the uncertainty associated with 
imputations. In total, 667 (2%) clinical data items were imputed. 

In our study, only a minority of patients underwent catheter 
based coronary angiography. An analysis restricted to patients 
who underwent catheter based coronary angiography could have 
been influenced by verification bias. 27 Therefore, we imputed 
data for catheter based coronary angiography by using the CT 
based procedure as an auxiliary variable, in addition to all other 
predictors. 28 Results for the two procedures correlate well 
together, especially for negative results of CT based coronary 
angiography. 16 " 18 This strong correlation was confirmed in the 
1609 patients who underwent both procedures (Pearson r=0.72). 
Since its data were used for imputation, the CT based procedure 
was not included as a predictor in the prediction models. Our 
approach was similar to using the results of CT based coronary 
angiography as the outcome variable when the catheter based 
procedure was not performed (which was explored in a 
sensitivity analysis). However, this approach is more 
sophisticated because it also takes into account other predictors 
and the uncertainty surrounding the imputed values. We imputed 
3615 (64%) outcome values for catheter based coronary 
angiography. Multiple imputations were performed using 
Stata/SE 11 (StataCorp). 

External validation of the Duke clinical score 

To evaluate the performance of the Duke clinical score, we 
calculated the predicted probability based on published 
coefficients 13 (that is, prediction of >75% stenosis). Since 
patients with evidence of previous coronary artery disease were 
excluded, we assumed all had a normal resting 
electrocardiogram. If resting findings were available and taken 
into account, any overestimation would increase further. We 
used a calibration plot to compare predicted probability with 
the observed proportions of severe disease (that is, >70% 
stenosis or >50% left main stenosis) in a calibration plot. 

Development of new prediction models 

We defined three prediction models: a basic model including 
age, sex, symptoms, and setting; a clinical model including age, 
sex, symptoms, setting, diabetes, hypertension, dyslipidaemia, 
smoking, and body mass index; and an extended model including 
all clinical variables and the coronary calcium score. Since all 
clinical variables are known to be associated with coronary 
artery disease, 29 all predictors were entered simultaneously in 
a multivariable, random effects, logistic regression model. We 
included hospital as a random effect to account for clustering 
of patients within hospitals. Availability of data for exercise 
electrocardiography was limited, and web appendix table 5 
explores the variable' s incremental predictive value. We omitted 
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non-significant predictors with small effects (that is, odds ratio 
<1.01). 

Setting variable 

To account for differences in patient selection across datasets 
(based on referrals to catheter based coronary angiography v 
CT based coronary angiography), we created a dummy variable 
for setting. This variable was coded "0" (low prevalence setting) 
if a patient came from a database that was created by selecting 
patients who underwent CT based coronary angiography (of 
whom only a proportion underwent the catheter based procedure 
in addition to the CT based procedure), and coded as "1" (high 
prevalence setting) if a patient came from a database that was 
created by selecting patients who underwent catheter based 
coronary angiography (of whom a proportion also underwent 
the CT based procedure). 

We intended to apply our prediction models to patients in low 
prevalence populations, for whom the best diagnostic 
management should be determined based on an estimated pretest 
probability. 1 " By contrast, all patients in the high prevalence 
setting had a clinical indication for catheter based coronary 
angiography, in whom estimating the pretest probability would 
not be relevant to them. However, because it would be inefficient 
to derive a prediction model in a low prevalence population 
only (since most patients will not undergo the reference 
standard), we also included databases with patients referred for 
catheter based coronary angiography. These data provided 
valuable information on the correlations between clinical 
presentation, risk factors, and the two angiography procedures, 
which was essential for reliable imputation of covariables and 
outcomes in the low prevalence populations. By including the 
setting variable, we could derive the model using all available 
data, and adjust for differences in patient selection. When 
applying the model for new patients with chest pain, the setting 
variable was set to zero. 

Predictor effects might differ across the low and high prevalence 
settings, and we tested these differences by using interaction 
terms between setting and all other variables. We also tested 
interactions between symptoms and sex, symptoms and age, 
and symptoms and diabetes. Linear effects of age and the log 
transformed coronary calcium score were tested by including a 
restricted cubic spline function with three knots (df=2). 30 31 

We quantified diagnostic performance by calculating the area 
under the receiver operating characteristics curve (c statistic). 
Reclassification was assessed by use of the continuous net 
reclassification improvement (web appendix table 2). 32 We 
regarded P<0.05 to be statistically significant. Analyses were 
performed using Stata/SE 11 (StataCorp). 

Validation 

We assessed the validity of the clinical model in a cross 
validation procedure. The four largest low prevalence databases 
with sufficient numbers for reliable validation, 33 and the 
remaining low prevalence databases combined, were each in 
turn removed from the model development sample. We then 
validated each model using the database that was omitted during 
model development. We calculated the c statistic and validated 
the model according to the steps in the box (the web appendix 
provides more detail). 26 30 31 



Results 

Data collection and study population 

We retrieved databases from 1 8 hospitals (table 1 and web 
appendix table 2). The study population included 5677 patients 
(3283 men, 2394 women; mean age 58 and 60 years, 
respectively). Nearly all patients (5190, 91%) underwent CT 
based coronary angiography, which revealed obstructive 
coronary artery disease in 1634 (31%). Of these 1634 patients, 
1083 (66%) underwent catheter based coronary angiography, 
which showed positive results in 886 (82%). Of the 3556 
patients without obstructive disease on CT based coronary 
angiography, 526 (15%) underwent catheter based coronary 
angiography, which showed negative results in 498 (95%). 
Overall, 2062 (36%) patients underwent catheter based coronary 
angiography, with 1176 (57%) diagnosed with obstructive 
coronary artery disease. Missing values occurred in four (0.1%) 
patients for age, six (0.1%) for symptoms, 126 (2.2%) for 
hypertension, 189 (3.3%) for diabetes, 187 (3.3%) for 
dyslipidaemia, 155 (2.7%) for smoking, 354 (6.2%) for body 
mass index, and 810 (14%) for coronary calcium score. 

Of the 3556 patients who did not have obstructive coronary 
artery disease revealed by CT based coronary angiography, 
3030 (85%) did not undergo the catheter based procedure. 
Results for catheter based coronary angiography were imputed 
for these patients, and were mostly negative (range 97-98.4% 
across the multiple imputations), which accords with the high 
negative predictive value of the CT based procedure. Of the 
1634 patients who had obstructive disease revealed by the CT 
based procedure, 55 1 (34%) did not undergo subsequent catheter 
based coronary angiography. For these patients, results for the 
catheter based procedure were imputed, and were mostly 
positive (range 65-77% across imputations), which accords with 
a reduced positive predictive value of the CT based procedure. 

External validation of the Duke clinical score 

External validation of the Duke clinical score overestimated the 
probability of severe coronary artery disease, as observed in our 
dataset (fig 1 !J and web appendix table 3). 

Development of new prediction models 

Table 2|| summarises the results of the random effects analysis 
in logistic regression and the continuous net reclassification 
improvement (web appendix table 4 provides more detail). The 
prediction models are available as an online probability 
calculator (http://rcc.simpal.com/NpfpV5; web appendix fig 
A2). In the clinical model, all predictors except body mass index 
were significantly associated with obstructive coronary artery 
disease. The clinical model improved the prediction, compared 
with the basic model (cross validated c statistic improved from 
0.77 to 0.79; table 2). Whereas an abnormal exercise 
electrocardiography had limited predictive value in the 
multivariable prediction model (web appendix table 5), the 
coronary calcium score was a major predictor, which increased 
the c statistic from 0.79 to 0.88 (table 2). Most predictor effects 
decreased after addition of the coronary calcium score; 
dyslipidaemia and smoking were no longer significant. We 
obtained similar results when using CT based coronary 
angiography as the outcome in patients who did not undergo 
catheter based coronary angiography. 

Validation 

Figure 2IJ and web appendix table 6 show the cross validation 
results for the clinical model. The c statistic ranged from 0.78 
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Box: Cross validation procedure 

Calibration-in-the-large 

When assessing the validity of a prediction model, the first step is to check whether the average prediction approximates the average 
observed outcome. This concept is referred to as "calibration-in-the-large". In this study, we compared the mean observed frequency of 
coronary artery disease in the validation data with the mean predicted probability. A negative value would imply that the mean observed 
proportion in the validation data is lower than the mean predicted probability. Conversely, a positive value would indicate that the mean 
observed proportion in the validation data was higher than the mean predicted probability. A statistically significant result indicates significant 
miscalibration, whereas a non-significant result supports validity of the prediction model 

Logistic recalibration 

The second step is to test whether the overall effect of the predictors in the model is valid for the validation data. The overall predictor effects 
in the validation data were re-estimated and compared with the reference overall predictor effects in the prediction model (reference is set 
to zero). A negative value would imply that the overall effects of the predictors in the validation data were lower than those in the prediction 
model. Conversely, a positive value would indicate that the overall effects of the predictors in the validation data were stronger than those 
in the prediction model. A significant result would indicate significant miscalibration of the predictor effects, whereas a non-significant result 
would indicate no difference in predictor effects, supporting the validity of the model 

Re-estimation 

The third step is to re-estimate the predictor effects in the validation data and calculate the difference (8 coefficients) with the estimated 
predictor effects in the prediction model (web appendix). 26 Non-significant differences (high P values) indicate no difference in the predictor 
effect, supporting the validity of the model 



to 0.81. The continuous net reclassification improvement was 
a measure of the relative change in the observed proportion 
when the predicted probability changes (web appendix), which 
was most favourable (102%) for the extended model compared 
with the clinical model (table 2). 

Assessment of calibration-in-the-large showed a significant 
difference between the mean observed outcome and the 
predicted probability (clinical model), for Azienda Ospedaliero 
Universitaria Parma, Rotterdam, and the combined low 
prevalence hospitals (fig 2). Logistic recalibration showed no 
significant differences between the overall hospital specific 
effects of the predictors compared with the overall effects of 
the predictors in the clinical model (fig 2). When re-estimated 
in specific datasets, the predictor effects were not significantly 
different from the predictor effects in the clinical model, except 
for the effect of typical chest pain for Azienda Ospedaliero 
Universitaria Parma. The results indicated that predictor effects 
were similar across datasets (web appendix table 6). 

Discussion 

Summary of key findings 

With recently collected data and modern statistical methods, 
we developed a prediction model that performed well in 
estimating the probability of coronary artery disease. The need 
for an updated model was evident by our results showing that 
the Duke clinical score significantly overestimated the 
probability of coronary artery disease. Age, sex, symptoms, and 
coronary calcium score were strong predictors for disease. The 
clinical model predicted probabilities between 2% for a 50 year 
old woman with non-specific chest pain without any risk factors, 
and 91% for an 80 year old man with typical chest pain and 
multiple risk factors. 

Previous publications 

In 1979, Diamond and Forrester" showed the importance of 
age, sex, and symptoms in the prediction of coronary artery 
disease. Despite its limitations, current guidelines still 
recommend the use of Diamond and Forrester's model. 7 8 As 
previously shown, the model tends to overestimate the 
probability of coronary artery disease, mainly in women, 14 and 
does not take into account cardiovascular risk factors associated 
with the presence of the disease. 

In 1993, Pryor and colleagues 12 13 developed the Duke clinical 
score in a large cohort of patients who underwent catheter based 
coronary angiography. The model predicted the presence of at 



least 75% stenosis, on the basis of age, sex, symptoms, history 
of myocardial infarction, smoking, dyslipidaemia, diabetes, and 
resting electrocardiography findings. In our study, we showed 
that the Duke clinical score also overestimated the probability 
of coronary artery disease. 

Limitations of the study 

The study population was derived from existing databases, some 
of which were designed for other research objectives (such as 
to investigate the diagnostic accuracy of CT based coronary 
angiography for coronary artery disease). In some studies, all 
patients underwent the reference standard test, whereas other 
studies selected patients for catheter based coronary angiography 
on the basis of results from the CT based procedure. When 
evaluating the diagnostic performance of CT based coronary 
angiography, selection on the basis of the test results could lead 
to verification bias or selective referral bias, which could affect 
the sensitivity and specificity estimates of such a test. However, 
in the current study, we did not assess the diagnostic value of 
the CT based procedure, thus making bias less likely. We also 
considered patients who underwent the CT based procedure 
only and not the catheter based procedure. We selected patients 
who underwent either angiography, or both procedures. 
However, not all patients presenting with chest pain will be 
referred for one procedure or the other, which limits the 
generalisability of our results. 

To also consider patients who did not undergo catheter based 
coronary angiography, we multiply imputed these results on the 
basis of results from the CT based procedure and all other 
covariables. Although data for the catheter based procedure 
were imputed for a large proportion of patients, we believe that 
the high sensitivity and specificity of the CT based procedure 
justifies the imputation; we also did not consider the CT based 
procedure as an explanatory variable in the prediction models. 
However, the CT based procedure could overestimate the 
severity of disease, which in turn could have overestimated the 
imputed severity of disease from the catheter based procedure. 
If bias was present in our models, it would tend to advocate 
further diagnostic investigation rather than missing the diagnosis. 

We combined existing data from several different hospitals. 
Since the current analysis was not the main purpose of the data 
collection, the selection of patients, availability of data, and 
predictor definitions differed across hospitals (web appendix 
table 1). Furthermore, some hospitals used quantitative coronary 
angiography to determine the degree of stenosis, whereas others 
used visual assessment. Overall, heterogeneity due to differences 
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between protocols, level of physician experience, and guideline 
adherence across hospitals could have influenced our results. 
Despite these limitations, the models presented had generally 
good discrimination (via the c statistic) and calibration. 

Because we intended to use our model in low prevalence 
populations, cross validation was performed using only the low 
prevalence datasets. The cross validation results in the data from 
Parma, Rotterdam, and the smaller hospitals combined, were 
less favourable in terms of calibration-in-the-large, possibly 
explained by heterogeneity. However, in general, calibration 
assessed graphically (fig 2) could be considered satisfactory, 
suggesting that the model is generalisable to other settings. 
Further external validation of our model in other populations is 
still needed. 

Our study focused on the prediction of obstructive coronary 
artery disease according to age, sex, type of chest pain, 
cardiovascular risk factors, setting, and coronary calcium score. 
Unfortunately, no data were available to assess the predictive 
value of findings based on other imaging tests (such as nuclear 
perfusion imaging, perfusion magnetic resonance imaging, and 
echocardiography). Furthermore, we were unable to show 
incremental predictive value of exercise electrocardiography, 
which may be explained by the limited availability of data. 

Finally, our analysis focused on the diagnostic prediction of the 
presence of coronary artery disease, defined as at least 50% 
diameter stenosis in at least one vessel, shown on catheter based 
coronary angiography. Predicting severe disease (for example, 
>70% stenosis in the left anterior descending coronary artery, 
three vessel disease, or left main coronary artery disease) would 
be helpful as a tool to select patients for revascularisation. 
However, the main purpose of the current analysis was the 
development of prediction models to help physicians select 
patients who would benefit from further testing. 

Future directions 

Other non-invasive imaging tests for the evaluation of patients 
with chest pain should also be considered as predictors for the 
presence of coronary artery disease. Furthermore, cost 
effectiveness analyses should be performed to establish 
appropriate thresholds for diagnostic testing, taking into account 
the long term benefits and harms. 4 34 

Clinical implications 

We showed that the Duke clinical score overestimates the 
likelihood of coronary artery disease, and we believe that our 
model improves the estimate of the pretest probability. By 
contrast with the Duke clinical score, we developed and 
validated our model using data from different hospitals, settings, 
countries, and included hypertension as a predictor. Finally, our 
model does not need inclusion of resting electrocardiography 
findings, which could be convenient in primary practice. 

A refined estimate of the probability of coronary artery disease 
allows doctors to make better decisions as to which diagnostic 
test is best in a particular patient, according to NICE guidelines, 
and to decide on further management based on the results of 
such tests. Our stepwise models can be used to evaluate the 
added value of performing coronary calcium score either before 
performing the test or after obtaining the calcium score. Our 
analysis shows that the coronary calcium score significantly 
improves the estimate of the probability of coronary artery 
disease, suggesting that the score should be considered for 
patients with chest pain. Use of the coronary calcium score is 
not routinely recommended by the guidelines from the American 
College of Cardiology/ American Heart Association 35 or the 



European Society of Cardiology, 9 whereas the UK guidelines 
recommend use of the score if the pretest probability is 
10-29%. 10 However, triage strategies using the coronary calcium 
score have been proposed 35 and are implemented in several 
centres. In this context, our prediction calculator could be used 
both to determine whether using the coronary calcium score is 
clinically useful (by checking whether the score alters the 
probability of coronary artery disease such that clinical 
management would change) and to determine the revised 
probability of coronary artery disease, based on the score's 
result. 

Our findings also suggest that the diagnostic value of exercise 
electrocardiography is limited (web appendix), which accords 
with its low sensitivity and specificity for detecting coronary 
artery disease, 36 and with explicit recommendations in the UK 
not to use the test to diagnose or exclude disease in patients 
with chest pain. 10 However, many physicians argue that the 
prognostic information obtained by exercise electrocardiography 
remains important for clinical practice. Alternatively, coronary 
calcium scores provide both diagnostic 37 3<) and prognostic 
information. 4043 

Prediction tools are useful only when they are easily accessible 
at the point of care, which is why we designed an online 
calculator (web appendix fig A2). The calculator could be 
implemented in electronic patient records, electronic order entry 
systems, or smartphone or tablet applications. Overall, prediction 
models that include age, sex, symptoms, and risk factors allow 
for accurate estimation of the probability of coronary artery 
disease in low prevalence populations. The addition of using 
the coronary calcium score improved the prediction of the 
disease. Implementation of these models could improve clinical 
outcomes, but would need further evaluation. 
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What is already known on this topic 

Current guidelines recommend use of the Diamond and Forrester model or Duke clinical score to estimate the pretest probability of 
coronary artery disease in patients presenting with stable chest pain 

The pretest probability of disease helps physicians identify which patients are most likely to benefit from further testing 

What this study adds 

The Duke clinical score overestimates the probability of coronary artery disease 

An updated prediction model for low prevalence populations was developed and validated using individual patient data from 1 8 different 
hospitals 

Use of the CT based coronary calcium score in the updated model improved the predicted probability of disease 
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Tables 



Table 1| Patient characteristics. Most commonly used risk factor definitions provided; characteristics data are no (%) of patients unless 
stated otherwise 





Low prevalence setting (10 hospitals) 


High prevalence setting (8 hospitals) 


Characteristic 


Value 


No (%) of patients with 
available data 


Value 


No (%) of patients with 
available data 


No of patients per hospital 


Mean (range) 


443 (80-1241) 


4426 (100) 


156 (85-549) 


1251 (100) 


Age (years) 


Mean (standard deviation) 


57.2 (12) 


4422 (99.9) 


63.6 (10) 


1251 (100) 


Interquartile range 


49-66 




57-70 




Range 


18-92 




18-93 




Sex 


Male 


2406 (54) 


4426 (100) 


877 (46) 


1251 (100) 


Chest pain* 


Typical 


759 (17) 


4424 (99.9) 


656 (53) 


1247 (99.7) 


Atypical 


2699 (61) 




278 (22) 




Non-specific 


966 (22) 




313 (25) 




Other clinical characteristics 


Diabetesf 


622 (15) 


4238 (96) 


229 (18) 


1250 (99.9) 


Hypertension^ 


2475 (58) 


4300 (97) 


840 (67) 


1251 (100) 


Dyslipidaemia§ 


2194 (52) 


4255 (96) 


801 (65) 


1235 (99) 


Smokingfl 


1231 (29) 


4273 (97) 


454 (36) 


1249 (99.8) 


Body mass index (mean 
(median))" 


28 (27) 


4117 (93) 


28 (27) 


1206 (96) 


History 


Family history of coronary artery 
diseasett 


1720 (44) 


3938 (89) 


136 (51) 


265 (21) 


Previous cerebrovascular 
diseaseiM: 


78(3) 


2531 (57) 


25 (9) 


269 (22) 


Previous renal artery disease 


43(1) 


3351 (76) 


2(1) 


316(25) 


Previous peripheral arterial 
disease 


79(2) 


3356 (76) 


10(3) 


369 (29) 


Exercise electrocardiography 


Normal 


671 (42) 


1612 (36) 


166 (30) 


547 (44) 


Abnormal 


443 (27) 




336 (61) 




Non-diagnostic 


498 (31) 




45 (8) 




Coronary calcium score§§ 


Mean (standard deviation), 
median 


160 (399), 4 


4009 (91) 


442 (643), 182 


858 (69) 


0 


1777 (44) 




155 (18) 




0-<10 


402 (10) 




44 (5) 




10-<100 


749 (19) 




154 (18) 




100-<400 


606 (15) 




208 (24) 




>400 


475 (12) 




297 (35) 




Results from CT based coronary angiographyHf 


No obstructive CAD 


3232 (75) 


4287 (97) 


324 (36) 


903 (72) 


Moderate CAD 


505 (12) 




501 (55) 




Severe CAD 


550 (13) 




78 (9) 




Results from catheter based coronary angiography*** 


No obstructive CAD 


406 (48) 


848 (19) 


480 (40) 


1214(97) 
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Table 1 (continued) 





Low prevalence setting (10 hospitals) 


High prevalence setting (8 hospitals) 


Characteristic 


Value 


No (%) of patients with 
available data 


Value 


No (%) of patients with 
available data 


Moderate CAD 


177 (21) 




541 (45) 




Severe CAD 


265 (31) 




193 (16) 




Patients who underwent both CT based coronary angiography and catheter based coronary angiography 


Results from CT based coronary angiography^) 


No obstructive CAD 


230 (31) 


742 (17) 


296 (34) 


867 (69) 


Moderate CAD 


277 (37) 




495 (57) 




Severe CAD 


235 (32) 




76 (9) 




Results from catheter based coronary angiography*" 


No obstructive CAD 


356 (48) 


742 (17) 


339 (39) 


867 (69) 


Moderate CAD 


172 (23) 




436 (50) 




Severe CAD 


214 (29) 




92 (11) 





CAD=coronary artery disease; moderate CAD=50-70% stenosis; severe CAD=>70% stenosis or >50% left main stenosis. 
'According to traditional chest pain classification. 19 

tDefined as fasting glucose levels of >7 mmol/L or treatment with diet intervention, oral glucose lowering agent, or insulin. 

$ Defined as blood pressure of >1 40/90 mm Hg or the use of antihypertensive drugs. 

§Defined as total cholesterol concentration of >5.2 mmol/L or treatment with lipid lowering drugs. 

Ulncludes current or past smoking. 

"Defined as weight (kg) divided by height (m z ). 

ttPresence of CAD in a first degree female relative (age <65 years) or male relative (age <55 years). 
ttHistory of carotid artery disease, stroke, or transient ischaemic attack. 
§§Agatston score as measured by computed tomography. 20 

KUSome hospitals only compared obstructive CAD (>50% stenosis) with no obstructive CAD (hospital numbers 2,3, and 1 8 in web appendix table 1 ; for example, 
they did not consider the severe category for coronary computed tomography angiography). One hospital did not categorise patients with >50% left main stenosis 
in the severe CAD category (hospital number 6 in web appendix table 1). 

""Two hospitals only compared obstructive CAD (>50% stenosis) with no obstructive CAD (hospital numbers 2 and 1 8 in web appendix table 1 ; for example, they 
did not consider the severe category for catheter based coronary angiography). 
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Table | Random effects logistic regression analysis* and cross validation, in the low prevalence setting 


Prediction model 




Basic 


Clinical 


Extended 


Age (per 10 years) 


1.89(1.74 to 2.04) 


1 .85 (1 .70 to 2.02) 


1.11 (0.99 to 1.25) 


Male sex 


3.89 (3.24 to 4.66) 


3.79 (3.13 to 4.58) 


2.19 (1.75 to 2.75) 


Chest pain (v non-specific chest pain) 


Atypical 


1.93(1.48 to 2.52) 


1 .88 (1 .44 to 2.46) 


2.05 (1.50 to 2.80) 


Typical, if diabetes is absent 


7.21 (5.64 to 9.22)t 


7.36 (5.64 to 9.61) 


7.57 (5.56 to 10.3) 


Typical, if diabetes is present 




4.91 (3.16 to 7.63) 


3.46 (2.12 to 5.63) 


Diabetes 




2.29 (1 .72 to 3.04) 


1.93(1.41 to 2.65) 


Hypertension 




1.40 (1.18 to 1.67) 


1.26 (1.04 to 1.54) 


Dyslipidaemia 




1.53 (1.25 to 1.86) 


1 .20 (0.95 to 1 .53) 


Smoking 




1.59 (1.30 to 1.93) 


1 .23 (0.97 to 1 .55) 


Coronary calcium score 


Log transformed (per standard deviation) 






4.69 (3.76 to 5.84) 


0-<10* 






2.23 (1.34 to 3.74) 


>10-<100* 






5.04 (3.38 to 7.52) 


>100-<400i 






15.3 (9.96 to 23.5) 


>400t 






35.9 (22.6 to 56.9) 


Cross validation (mean§) 


C statistic 


0.77 


0.79 


0.88 


Net reclassification improvement (%)1) 




35 


102 



Data are odds ratio (95% confidence interval) unless stated otherwise. All odds ratios showed significant associations (P<0.05) apart from age, dyslipidaemia, and 
smoking in the extended model. 

•Random effect for hospital included to account for clustering of patients within hospitals. Body mass index omitted from all analyses because odds ratio was less 
than 1 .01 and was non-significant. Setting and the interaction between diabetes and typical chest pain, and between setting and coronary calcium score were 
predictive and were included in all models. All other interactions were not significant. Test for a non-linear effect of age was not significant. Evidence indicated 
additional non-linear effect of coronary calcium score beyond the log transformation, which was considered not clinically important, and omitted for simplicity. 
-(-Irrespective of diabetic status, since basic model does not include diabetes. 

^Separate analysis using coronary calcium score as a categorical variable, adjusted for all other predictors in the model, reference category is score 0. 
§Mean of the cross validation procedures calculated using the four largest low prevalence datasets and remaining low prevalence datasets combined. 
HCalculated by comparison with the next model on the left; defined as weighted sum of the increase in observed proportion among patients whose predicted 
probability goes up, and decrease in observed proportion among those whose predicted probability goes down (web appendix). 23 
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Figures 



C statistic (95% CI) 0.78 (0.76 to 0.81) 
Calibration-in-the-large -2.361 P<0.001 
Recalibration slope -0.323 P<0.001 
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Fig 1 Calibration plot of the Duke clinical score, in low prevalence datasets (n=4426). Distribution of predicted probabilities 
shown separately for patients with and without severe coronary artery disease. Triangles indicate observed proportions of 
severe disease, by tenths of predicted probability; 95% CI=confidence interval 
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Parma (n=1241) 

C statistic (95% CI) 
Calibration-in-the-large 
Recalibration slope 



Miami (n=82l) 



0.78(0.74 to 0.81) 
0.334 P=0.001 
-0.117 P=0.14 
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C statistic (95% CI) 0.80 (0.74 to 0.85) 
Calibration-in-the-large -0.046 P=0.76 
Recalibration slope 0.046 P=0.75 



















Innsbruck (n=668) 

C statistic (95% CI) 0.78 (0.73 to 0.82) 

Calibration-in-the-large 0.113 P=0.33 

Recalibration slope 0.023 P=0.85 
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Rotterdam (n=47l) 
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Fig 2 Validity of clinical model using the four largest low prevalence datasets and the smaller remaining low prevalence 
databases combined. Distribution of predicted probabilities shown separately for patients with and without obstructive 
coronary artery disease. Triangles indicate observed proportion of disease, by tenths of the predicted probability; 95% 
Cl=95% confidence interval 
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